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Abstract 

In this paper, a new face recognition technique based on 
Hidden Markov Model (HMM), Pre-processing, and 
feature extraction (Canny) is proposed. Two main 
contributions are presented. In the pre-processing, the 
input image’s edge is fixed to five pixels and spacing 
between various parts of the pixels is fixed to a trial 
threshold. The second contribution is a new technique to 
extract the image's features by splitting the image into 
non-uniform height depending on the distribution of the 
foreground pixels. The foreground pixels are extracting 
by using the vertical sliding windows. The pre-processing 
steps have enhanced the performance of the system, for 
example, speed is the essence. In this paper, Hidden 
Markov models alongside the canny operator which 
proved high accuracy recognition of the face and its 
boundary. The presented technique is faster than some 
other techniques that are investigated for comparison. In 
addition to that, it reveals, its ability to recognize the 
normal face and face boundary very efficiently 

Keywords: HMM, Canny operator, Face 

Recognition, Accuracy 

Nomenclature 


DCT 

Discrete Cosine Transform 

HMM 

Hidden Markov Model 

LDA 

Linear discriminate Analysis 

ORL 

Olivetti Research Laboratory 

PCA 

Principal Component Analysis 

PDBNN 

Probabilistic Decision- Based Neural Network 

SVD 

Singular Values Decomposition 

Wo 

represents the width of overlapping of the 
vertical frames 


1. Introduction 

Many up to date techniques have been immerged during 
the recent years, due to the urgent need for security issues 
and in particular the pattern recognition. In digital image 
processing, edge detection [1] is considered the main 
element of recognition method. Therefore, it is a 
necessary for image analysis to extract its landmarks, 
forms, and then convert it into data that can be processed 
Face recognition has become necessary in commercial, 


law and security researches. Recently, a dedicated work 
targets face recognition process that aims to handle the 
relation between time and accuracy. However, face 
recognition faces too many challenges such as the 
difference between human faces in color, shape, and 
dimensions. Thus, a demand for image analysis has 
appeared to extract its landmarks and forms, and then 
convert it into data to be processed. A different number 
of models can be used for addressing this issue. Such as, 
stochastic process is the hidden Markov model. There are 
two main features used for face detection. The first one is 
the feature based where analysis is performed on the 
main geometry of the face like eyes, nose, ears, and 
mouth. The second feature is a holistic base where the 
face is analyzed as a two-dimensional pattern. The 
available researches have followed a certain techniques 
for data reduction and feature extraction, which is used 
for face detection, A wide range of models can be used 
for addressing this issue, most commonly models 
are Geometrical Feature Matching [2], Eigenfaces [3], 
Neural Networks [4], Linear discriminate analysis (LDA) 
[5], Principal component analysis (PC A), Discrete cosine 
transform (DCT) [6] and the fuzzy logic technique [7], 
which is used for face detection and Hidden Markov 
Models. Geometrical feature matching model is based on 
extracting geometrical features to form a complete 
picture of the face. This model showed a low accuracy 
with 75% on ORL database recognition rate. Another 
model is Eigenfaces method which uses the 
Principal Component Analysis (PC A) with 90.5% 
recognition rate on ORL database. Although this model is 
simple and fast however it is not robust enough over the 
changes in face orientation and lighting conditions. 
Hidden Markov Model is used for developing features 
extraction methods and enhancing not only the operation 
of face detection but also the data processing time as will 
be shown in this paper. In this paper we introduce a new 
approach for preprocessing the input image and a new 
method of extracting features by splitting the input image 
vertically with non-uniform heights. This paper is sorted 
as follows. In section 2, the proposed (presented) 
technique is explained. In section 2.1, the pre-processing 
step is presented. In section A, the feature extraction 
followed by recognition system with the use of HMM is 
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introduced in section B and C. In section 3 simulation 
results are described. In section4 discussion is presented 
Finally, conclusions are highlighted in section 5 

2. Material and Methods 

Material used in our proposed recognition technique: 

• Available subsets of the benchmark ORL face 
database. In our technique validation, we used 80 
images for training purpose (8 persons x 10 
images), and 40 images for testing purpose (8 
persons x 5 images). The face recognition 
technique presented in this paper was developed, 
trained, and tested. 

• Using Matlab 7. 10, the computer was a Windows 7 
machine with a 2.1 GHz Intel core 2 processor and 
2.87 GB of RAM. 

2.1. Methodology 

Our proposed Technique consists of multiple steps: 

• The pre-processing step which will be explained in 
this section. 

• The feature extraction to prepare data to be 
processed. 

• Appling Canny operator 

• Appling HMM on the recognition system. 

• We calculate the accuracy (recognition rate) to 
measure the ability of recognize the image by the 
system.Figurel shows the block diagram for our 
proposed technique. 



Pseudo code 

READ new input image 
notknown = 1 
WHILE (not known) 

DO median filter to reduce noise 
DO other enhancement 
DO Canny to detect edges 
IF (imagesize! = threshold size) 

DO resize the image to new size 
ENDIF 

DO segmentation of the resized image with 
overlapping sliding window (horizontal and 
vertical) 

DO extract features from the segments of the image 
IF (HMM training) 

IF (new class) 

DO add new class model 
ELSE 

DO modify HMM class model using the image 
ENDIF 

ELSE 

Classify the image using the HMM models 
IF recognized 

WRITE output the image with decision 
Break from while loop 

ENDIF 


ENDIF 
END WHILE 


Figure 1 Face Recognition Proposed Technique 


A. Pre-processing 

The image from RGB format is change to analogous 
grayscale format, as a first step. A grayscale image is a 
monochrome image, which contain data related to 
brightness only without containing any information about 
the color. A grayscale data represents a broad range of 
intensities. The typical image contains 8 bit/pixel, this 
lets the image to be represented with different level of 
brightness (0-255) [8]. The pre-processing is mandatory 
step to enhance the ease of features extraction and reduce 
the noise in the image, by using the median filter. The 
pre-processing step includes changing of the image’s 
edges to a settled pixel’s numbers ,that occasionally 
decrease the space between parts of pixels, if this space 
is larger than a calculated threshold. In pre-processing we 
change the input image’s edges, There are contradictions 
in the image’s edges in the database. After several trials, 
it was found that a rise in the recognition rate after 
normalizing the input image’s edges and pixels, that goes 
through two steps: 

• Change the input image’s edges as shown in 
Figure.2.b; after changing, the image’s edges in the 
original are thinner than others of the same image 
as shown in Figure 2. a. After thinning, the image’s 
edges of each will have become the same pixels (2 
pixels) as shown in Figure 2.c. 

• Spread the binary image by using 4-pixels framing 
the original pixel, as shown in Fig. 2.d. So, the two 
image’s edges will become the same. 
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In pre-processing, we decrease the space between pixel’s 
parts to find the places where no pixels exist so the 
vertical projection is used, after that calculates the width 
(d) of each place, as shown in figure 3 a. If the width is 
larger than a threshold (8 pixels, by trial and error), then 
removing (d-threshold) to get a stable separation between 
of pixel’s parts as shown in Fig.ure 3.b 



(*) Urigmaf image 0>) 

i 

*; - 

I 

(0 

Thinning 




Spreading 


Figure.2 The input image’s edges are changing, in (a, b) the 
identical original image, which the edges are not the same, (c) 
After thinning the edges, (d) After spreading to fix the image’s 
edges. 



(a) 



Figure 3 (a) Original image with large space between pixel’s 
parts, (b) Decreasing the space between pixel’s parts. 


B. Feature Extraction 

After the pre-processing stage, the binary format of the 
input image as represented in the ORL database, which is 
used to get a feature vector. Identified the pixel’s 
background "zero" and the foreground "one". The image 
is split into n horizontal segments, which have an 
identical number of the pixel located in the foreground in 


each segment. The total pixels of the image are counted 
and divided by n, to get the exact number of pixels per 
segment. The image is split into vertical frames 
(overlapping), each frame is split into fixed-width small 
cells, that the cell's height reliant on the distribution and 
the pixels of the foreground in the images. In the image, 
the sliding window is moved from right to left and 
determines the features for each frame, as shown in 
Figure 4 


A 


height (h) 


Figure.4 Using the sliding window 

The image’s height represents (h), the image’s width 
represents (W), (Wo) represents the width of overlapping 
of the vertical frames, and (Wv) represents the width of 
the vertical frame. The best scenario in our technique 
when ‘Wv - 7 pixels with ‘Wo - 4 pixels, this sliding 
accomplished the 100% accuracy (recognition rate) as it 
extracted the maximum necessary features for 
recognition. As it extracted the maximum necessary 
features for recognition. 6 cells (trial and error) are used 
per vertical frame with non uniform heights to extract 
this feature for each cell, the number of "1" pixels are 
calculated to produce 6 features per frame (6 cells per 
frame xl feature per cell) as shown in figure5. 
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• Canny Edge Detection 

Advantage of Canny: 

- Good detection accuracy 

- Good localization: 

- Single response constraint 

Canny edge detection can be applied using the following 
five steps [9]: 

1) Removing the noise and smooth the images by 
applying Gaussian filter. 

2) Finding the intensity gradients of the image. 

3) Getting rid of spurious response to edge detection 
by applying non-maximum suppression. 

4) Determine potential edges by applying double 
threshold. 

5) Track edge by hysteresis: Remove all weak edges 
that are not connected to strong edges to finalize the 
process of detection. 

Detector eliminates noise by smoothing the image, and 
then it finds the gradient to show high spatial derivatives 
regions [6]. In these areas the canny edge detector 
performs tracking and eliminates any pixel that is not at 
the maximum. [7]. 

Algorithm 

1- Compute fx and fy 

f x = ^-(f*G) = f*^-C = f*Gx 

OX ox ^ 

fy = ^(f*G) = f*^G = f*Gy 

oy °y 

G(x, y) is the Gaussian function, Gx(x, y) is the 
derivate of G(x,y)with respect to x : 

Gx(x, y) = — ^ G(x, y) ( 2 ) 

(7 

Gy(x,y) = -^-G(x,y) (3) 

(7 

2 - Compute the gradient magnitude: 

magnii, j ) = ^jf 2 + f 2 (4) 


3- Apply non-maxima suppression and hysteresis 
thresholding/edge linking. 

C. Hidden Markov Model 

HMM process is a complex stochastic process. This 
process is repeated twice to form a well- defined model 
for the sequential data. HMM is a defined set of states, 
and transition probabilities control the transitions among 
those states. HMM consists of a different number of 
states and transition between those states. These 
transitions are controlled by transitions probability 
distribution. HMM process considers the feature 
observations in every state, those to produce the 
confidence of the modeled object according to its 
probability density function. From the unknown pattern, 
the classification can be obtained according to the 
extracted feature by searching a model that produces a 
maximum probability [10]. In this paper, HMM is discrete. 
In HMM Viterbi algorithm is searching for the best 
sequence of a hidden state given the input feature vector. 


Three states HMM are represented in fig. 5, which shows 
that we let a move to the current and the following states 
only. Two of these are emitting states and have their 
output probability distributions [11]. The transition 
matrix for this model is [3 x 3], which represent the 
transitions between these states such that in the case of 
transition, we will set the matrix element to one and 
matrix element will be set to zero in the case of no 
transition. 


Cl2,2 Cl3- 3 
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Figure 6 Three states of transition from left to right 


The element of HMM are: 

• N is the number of state in the model. If S is the set 

of state, then S = {Si,Si,S 3 , SVj.The state of the 

model at time t is given by (ft e S,l ^ t < T , where 
T is the length of the observation sequence 
(number of frames). 

• M, the number of different observation 

symbols. If V is the set of all possible 
observation symbols V= jvi, V2, , Vm} . 

• A the state of transition probability matrix 
i.e. A= | ay} where 

ciij — P[qt = Sj \qt-\ — Si\ (5) 


1 < i, j < N with the constraint , 0 < at, j < 1 


and, 



1 <i<N 


• B, the observation of symbol probability matrix i.e. 


B— \bj(k)} , where 


bj(k) = P[Ot = Vk\qt = aS/] (6) 

l<j<N,l<K<M and Ot is the observation 
symbol at time t . 

71, the initial state distribution, i.e. = j;r/} where 


7 ii=P[qi=Si],l < i < N (7) 

Using a notation, a HMM is defined as the triplet 
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A,-(A,B,7l) (8) 

the above characterization corresponds to a discrete 
HMM. In a continuous density HMM, the state are 
characterization by continuous observation density 
functions. The most general representation of the model 
probability density function (PDF) is a finite mixture of 
the form: 

M 

bi(0)=X CikN (O , JUik , U ik) (9) 

k = 1 

where, Cik is the mixture coefficient for the Kth mixture 
in state i. without losses of generality N(0, /Uik , U ik) is 
assumed to be a Gaussian PDF with mean vector ]Uik and 
covariance U ik 

3. Simulation Results 

The following parameter has been determined through 
validation: Number of states. Figure6 shows the samples 
of ORL face database. The following table 1 the states 
and effect of changing time versus accuracy (recognition 
rate). Table 2 and 3 shows the comparison with different 
techniques in recognition rate and Training Time. 
Figure7 shows the sample of ORL database Figure8, 9 
shows the effect of the noise and PSNR. Figure 10 shows 
the effect of the given data in table 1, Figured 1 shows the 
proposed technique is proved to be fast with respect to 
some other techniques that are investigated for 
comparison. Table 4 the proposed method which tested 
on images size of ORL database where the other methods 
use the same images size and measuring the error 
percentage after testing process. 


PSNR vs Noise variance 



noise density 

Figure 9 PSNR before and after different filters. 


Table 1: shows the effect of timing and states on the recognition 
ra te (accuracy) results 


#sample 

Average 

Time/Image(sec) 

Recognition 
rate(Accuracy %) 

1 

0.00331 

80.4 

2 

0.00336 

83.3 

3 

0.00339 

85.1 

4 

0.00346 

86.6 

5 

0.00351 

88.3 

6 

0.00360 

92.3 

7 

0.00366 

94.2 

8 

0.00370 

98.2 

9 

0.00372 

98.5 

10 

0.00376 

99.2 

11 

0.00380 

100 

12 

0.00381 

100 

13 

0.00381 

100 



Figure 7 Sample of ORL face database 

Original image Noisy image with salt and pepper (noise density = 0.03) 



Filtered image with average filter Filtered image with median filter 



«0 


Figure 8 The effect of noise in sample ORL face database 
a-Original Image b- Gaussian noise image 
c- Average filter d- median filter 


Time(s) 



^^Time(s) 


Accuracy 


Figure. 10 Times (ms) versus the Accuracy 


Table2 Comparison Results of Different Face recognition 
alg orithms 


Models 

Recognition rate 
(%> 

Proposed Technique 

100 

E-HMM 

99.16 

Eigenfaces 

85.9 

Fisherfaces 

92.3 

Laplacianfaces 

93.2 

ML-HMM 

88.2 

MCE-HMM 

93.3 

MC-HMM 

97.5 
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Our proposed technique gives the highest recognition 
accuracy for all the sample sets in training and testing 
images. Comparison result are shown in figure 11 and 
Table3. 



Figure 1 1 Comparison between different techniques 


Table 3 Comparison ORL database with different technique in 
Recognition rate and training time 


Model 

Recognition 
Rate (% ) 

Training 
Time/ image 

Proposed Technique 

100% 

0.00300 sec 

PDBNN 

96% 

20 min 

n-tuple 

86% 

0.9 

1DHMM + Wavelet 

100% 

1.13 sec 

Pseudo-2D HMM 

95% 

1 sec 

DCT-HMM 

99.5 % 

23.5 sec 

ID HMM + SVD 

99% 

0.63 sec 


The Comparison with different face recognition 
techniques and the proposed technique on the ORL face 
database. It is important to notice that all different face 
recognition techniques in Table 3 use 112^92 resolution 
of ORL face database where we use 64x64 image size. 


Table 4 Comparative results on ORL database, which is tested 
on images size of ORL database where the other methods use 
the same images size. 


Method 

Error 

Proposed Technique 

0% 

Top-down HMM + gray tone features 

13% 

Eigenface 

9.5 % 

Pseudo 2D HMM + gray tone features 

5% 

Elastic matching 

20% 

PDNN 

4% 

Continuous n-tuple classifier 

2.7% 

Top-down HMM + DCT coef. 

16% 

Point-matching and correlation 

16% 

Ergodic HMM + DCT coef. 

0.5% 

Pseudo 2D HMM + DCT coef. 

0% 

SVM + PCA coef. 

3% 

Independent Component Analysis 

15% 

Gabor filters + rank correlation 

8.5% 

Pseudo 2D HMM + Wavelet 

0% 

Markov Random Fields 

13% 

ID HMM+SVD 

1% 


— ■— Proposed Tech. 1 

— •— PCA 

-#-2D-PCA 

• • KPCA 

• • •• • PCA+SVM 

— A— SVM 

MLP 
RBF 

LBP-5x5 

— +• LBP-7x7 
LBP 7x7w 

1 .train 2_train 3_train 4_train 

Figure 12 Comparison between different techniques 

It is worth mentioning the technique has been 
implemented without performing the pre-processing step 
mentioned in section A. The results for different sets of 
databases showed that the recognition rate is about 97%. 
This implies clearly the importance of the pre-processing 
on the accuracy (recognition rate). 

4. Discussion 

The recognition rate of the proposed technique in this 
paper is higher than that of the traditional algorithms 
based on HMM. Also, we compare our proposed 
methodology with different face recognition methods 
such as EigenFace, FisherFace, Laplacianfaces, ML- 
HMM, MCE-HMM, and MC-HMM [12], our proposed 
method gives the highest recognition accuracy for all the 
sample sets (training and testing). Comparison result is 
summarized in the Table2, 3 and 4 and Figure 11. 

It is worth mentioning the technique has been 
implemented without performing the pre-processing step 
mentioned before. The results for four different sets of 
databases showed that the recognition rate is about 97%. 
(I.e. Images in ORL database passed through several 
stages: 

1 - Improve the signal to noise ratio by using the filters 
(average and median filter). The PSNR before pre- 
processing at different faces shown in figure8 

2- Normalizing the input image’s edges and pixels, by 
changing the input edges. 

3- Decreasing the space between the pixel’s parts. 

This implies clearly the importance of the pre-processing 
on the accuracy (recognition rate). Feature extraction 
played important rule in canny edge detection to extract 
the features from images and remove the noise. 

HMM's for classification the standard approach is: 

• Separate the data into the data sets for each 
class. 

• Train one HMM per class. 

• On the test set compare the likelihood of each 
model to classify each window. 

Supervised training: use statistics of known samples. 

• Trainer knows underlying sequence for samples. 

• Estimates transition/emission probabilities from 
samples database. 

Unsupervised training (learning): update HMM 
parameters based on new samples. 
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5. Conclusions 

In this paper, we introduced a new technique for image 
recognition. We proposed our technique based on 
intelligent features using efficient operator (Canny). The 
major contribution of the presented technique is the pre- 
processing stage. In the pre-processing, the input image’s 
edge is fixed to four pixels and spacing between various 
parts of the pixels is fixed to a trial threshold. The pre- 
processing steps have enhanced the performance of the 
system by about 3 % as explained previously. Although 
pre-processing has simple and important steps but it 
simplifies the features and enhances the recognition rate. 
Our proposed technique showed the best performance 
compared with the other tested techniques in the sense of 
speed and recognition rate. 
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